Interactive multimedia communication system

ABSTRACT

A system and device is provided that enhances the interactivity of multimedia information across a communications network or in combination with other networks including telecommunication and cable broadcasting. The system includes a plurality of multimedia call processing system, an interactive call processing system, and a plurality of interactive multimedia devices (IMDs). This system has the advantage of providing compression and/or transmission algorithms to maximize enhancement of the multimedia information.

FIELD OF THE INVENTION

The present invention relates to a method and system for the production,transmission and reception of multimedia information over acommunication network and more particularly to the production,transmission and reception of multimedia information with enhancedinteractivity over such a network.

BACKGROUND OF THE INVENTION

Multimedia information can be transmitted in a variety of applicationsover a network. What is meant by multimedia information is informationthat has several parameters which can be enhanced or diminisheddepending upon the importance of the particular information beingconveyed. For example, in a video environment there may be foregroundand background information that is to be provided to a screen. Thescreen can typically be a video screen or a computer screen. The mostcommon example of that type of system is an interactive video systemsuch as seen on cable television networks (i.e., QVC or Home ShoppingNetwork). These types of systems allow for a viewer to look at thescreen and pick the type of product that he/she may want based onreviewing the information on the screen and then calling a particulartelephone number to purchase the product.

Multimedia information can also be audio information in which news,weather, music and like could be transmitted with the use of backgroundinformation such as white noise, background music or other informationto complete the transmission. Multimedia information can also be acombination of graphics, video and music in an entertainment form suchas Karaoke. As has been above mentioned, there are interactive systemspresently available that allow for individuals to access certain video,audio and graphics information on a network. All presently known systemshave the problem of requiring a significant amount of bandwidth to allowfor the quality transmission of the multimedia information.

It is known that multimedia information can require a significant amountof bandwidth. In the case of video information as much as 30 MB/sec.bandwidth is required to send a quality video image along a transmissionpath. In the case of audio information, as much as 10 MB/sec. bandwidthis required to allow for high quality audio signal to be producedaccurately.

What is meant by limited bandwidth can be viewed in two different ways.One way is to view the total available bandwidth at any instant in time.Another way of deciding what the particular bandwidth requirement is bydetermining the bandwidth that is available over a specified timeperiod.

The available bandwidth therefor can be limited in two ways; first, theoverall bandwidth of the network is limited such as in telephonenetworks where the bandwidth is approximately ten (10) kilohertz.However in this example, the available bandwidth over a specified periodof time could be very high because a particular telephone set is notutilized constantly.

On the other hand, in looking at a large bandwidth network such as cabletelevision, although the bandwidth is significant, most of the availablebandwidth is utilized for providing the network signals. In such anexample, the available bandwidth over a specified period of time mightbe much less than the above-mentioned telephone system.

Telephone networks have traditionally been used to transmit data orvoice information. There are telephone sets that can also transmit videoinformation over telephone lines. For example, American Telephone &Telegraph (AT&T) has introduced a phone called the Videophone which cantransmit video images across the telephone network. It is becomingreadily apparent that the telephone networks can be used for interactivemultimedia information also. However, as has been mentioned above atypical telephone network has a limited bandwidth to transmit certaininformation. Particularly when attempting to transmit video informationwhich might require significant bandwidth the telephone lines areinadequate to provide quick transmission of a high quality video image.

In addition, in such systems the communication network, such as atelephone network itself, will not provide the enhancement capabilitiesrequired to provide the psychographic information that will enhanceinteractivity. Finally, in a typical telephone network many of thetechnologies required for such a system has not entered the home. Forexample, although there are many known video enhancing capabilities,audio enhancing, printing, telephone call processing control systems,and the like in commercial use, those techniques are not typically partof the home environment.

Hence, what is needed is an interactive multimedia system that provideshigh quality information over a telephone network. What is also requiredis a system that will allow for the production, transmission, andreception of interactive multimedia information that includes enhancedinteractivity. What is also needed is a device that can be utilizedwithin the home that can receive and transmit enhanced multimediainformation and control a number of other devices for useful purposes.

It is important that such a device be inexpensive to install in the homeenvironment. Furthermore, it is important that the interactivemultimedia device be one that when installed does not require themodification of the existing network.

The present invention provides such a system and device to be utilizedin conjunction with a communication network such as a telephone networkor the like.

SUMMARY OF THE INVENTION

An interactive multimedia communication system is disclosed that can beutilized with a telephone network, a similar switched network or incombination with a broadcast network such as satellite or cable.

The interactive multimedia communication system comprises interactivemultimedia mastering system means for receiving program materials from aprogram source; multimedia call processing system responsive tointeractive multimedia mastering system program materials for providinginformation; means interactively responsive to the interactivemultimedia mastering system means and the multimedia call processingsystem for controlling the flow of multimedia information to themultimedia call processing system; and a plurality of interactivemultimedia devices for receiving and transmitting multimedia informationto and from the multimedia call processing system.

In another facet of the invention, the interactive multimedia devicecomprises a multimedia modem means for receiving interactive multimediainformation from the telephone network; memory means for receivinginteractive multimedia information from the multimedia modem means;means for compressing at least a portion of the interactive multimediainformation received from the multimedia modem means and the memorymeans; means for decompressing at least a portion of the interactivemultimedia information; means for transmitting the uncompressed portionof the interactive multimedia information received from the telephonenetwork; mixing the decompressed portion with the uncompressed portionand providing an output; means for providing the output of theinteractive multimedia information to a display; and means forinteractively controlling the multimedia modem means, compressing means,decompressing means, mixing means and providing means to provideinteractive multimedia information to the display. In one facet of thepresent invention there are means to control other devices to performspecific functions including printing and the control of householddevices.

Through such a system and device a system can be utilized with anexisting telephone network to produce high quality multimediainformation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a block diagram of a prior art interactive multimedia system.

FIG. 2 is a block diagram of an interactive multimedia system for use ina telephone network alone or in combination with a cable tv broadcastfeed in accordance with the present invention.

FIG. 3 is a flow chart of a first embodiment of the optimization methodwhich is utilized in a telephone network in accordance with the presentinvention.

FIG. 4 is a flow chart showing the operation of a second embodiment ofan optimization method which is utilized in a telephone network inaccordance with the present invention in which a matrix of compressionand transmission algorithms are utilized.

FIG. 5 is a flow chart showing psychographic enhancement in accordancewith the present invention.

FIG. 6 is a chart showing the cooperation of a compression algorithmwith a transmission algorithm in accordance with the present invention.

FIGS. 7A and B is a block representation of digital information of animage file and a MIDI file.

FIG. 8 is a block representation of the multimedia call processingsystem (MCPS) in accordance with the present invention.

FIG. 9 is a block diagram of a general embodiment of an interactivemultimedia device (IMD) in accordance with the present invention.

FIG. 10 is a block diagram of a second embodiment of an IMD in which theIMD is connected to a printer in accordance with the present invention.

FIG. 11 is a block diagram of a third embodiment of an IMD in which theIMD produces a facsimile in accordance with the present invention.

FIG. 12 is a block diagram of a fourth embodiment of an IMD in which theIMD is connected to a facsimile machine in accordance with the presentinvention.

FIG. 13 is a representation of a remote control utilized in conjunctionwith the system architecture of the present invention.

DETAILED DESCRIPTION OF THE INVENTION

The present invention relates to an improvement in the transmission andreception of multimedia information when utilizing existing networks.The following description is presented to enable one of ordinary skillin the art to make and use the invention and is provided in the contextof a patent application and its requirements. Various modifications tothe preferred embodiment will be readily apparent to those skilled inthe art and the generic principles and features described herein.

Referring now to FIG. 1, what is shown in simple block diagram form is aprior art interactive multimedia system 10. In such a system 10, asender sends and receives information on one side of system 10 and thereceiver sends and receives information on the other side of the system10. The system 10 can be a single device such as a CD-ROM player or anetwork capable of fully interactive (2-way) multimedia transmission.

Before the present invention is described in detail certain backgroundinformation should be discussed to clarify the utility of the presentinvention in terms of existing multimedia systems. In a multimediasystem, various sensory information is provided to a receiver. In thecase of video information, this information takes the form of foregroundand background images that display a particular scene. In the case ofaudio information, the foreground and background signals are such thatthe foreground information is speech and the background information ismusic.

Typically, multimedia systems in whatever form provide this informationover a single transmission line. In so doing, the amount and quality ofthe multimedia information is severely limited by the bandwidth of thetransmission line.

The present invention first differentiates between important and lessimportant multimedia information by separating the information intoprimary and secondary layers through the use of a program model tominimize the bandwidth limitations. In such a system each layer willhave its own set of parameters that are important psychographically,however, the secondary layer will not vary as much as the primary layer.

What is meant by a program model refers to psychographic parameterswithin the multimedia system, that is parameters that relate to anindividual's sensory perceptions when encountering multimediainformation. These parameters comprise a set of unique and consistentelements for a particular class of multimedia information. In accordancewith the present invention, its multimedia information is separated intodifferent layers in accordance with the program model. Therefore, by wayof example, in the instance of video images, the foreground andbackground information might be divided into different layers.Similarly, in the case of audio information, the news information,weather information, or the like may be one layer whereas the backgroundmusic may be the other layer.

In the present invention these layers will be divided into primary andsecondary layers in accordance with the information's importancerelative to the program model. The most important information isidentified and enhanced to provide the best quality information to thereceiver of the multimedia information.

In the preferred embodiment, the primary layers will be enhanced in sucha way to provide a perceived improvement in quality of the multimediainformation presented. In one embodiment the secondary layers arepresented that may or may not be enhanced. Thereby the importantinformation or the primary layers that are to be transmitted can beidentified and selectively enhanced in accordance with the presentinvention.

In addition, the primary layers generally can be enhanced throughcritical psychographic parameters take the form of spatial, color,audio, and temporal variables that occur in the primary or secondarylayers.

In a communication system such as a telephone network it is importantthat the multimedia information that is produced, transmitted andreceived is enhanced in some manner. This is necessary to ensure thathigh fidelity, high quality information is presented to the viewer.Therefore, it will be possible to bring a superior product into the homeenvironment. It is known, for example, it has been possible to providevideo information over the telephone lines. However, it has been aproblem sending high quality video information due to the bandwidthrequirements that are needed to provide such high quality videoinformation.

The present invention is directed towards a method and apparatus forenhancing the interactive multimedia information transmitted across anetwork. To more fully explain such a method and apparatus, refer now toFIG. 2 which is a block diagram of the overall system architecture 40for an enhanced interactive multimedia system. The overall architecture40 would be connected to a telephone network or the like so as toreadily access the transmission lines located therein. Referring now toFIG. 2, the system architecture 40 comprises an interactive multimediamastering system (IMM) 42 which receives program source material. TheIMM system provides information from the telephone network to amultimedia call processing system (MCPS) 44 and an auxiliary system 46.The multimedia call processing system 44 will provide and receiveinformation from interactive multimedia devices (IMDs) 52. The structureof the IMDs 52 will be described in detail later in this specification.These IMDs 52 can take on many different forms dependent upon how muchintelligence is located in an IMD 52 relative to the system architecture40.

The system architecture 40 provides program material which will enhancethe interactivity of information that is transmitted along the telephonenetwork. Accordingly, what the system architecture 40 represents, infact, is an additional network which would receive information from theprogram source which would also be part of the existing telephonesystem. The system architecture 40 will then be utilized to provide forenhanced multimedia information through psychographic manipulations orother enhancements to the systems to provide for an improved interactivemultimedia system.

The system 40 as above-mentioned includes an IMM 42 that provides muchof the optimization of the program source material for presentation toMCPS 44.

To more specifically describe this optimization technique, refer now toFIG. 3 which is a block diagram of an optimization method in accordancewith the present invention. This optimization technique has beendescribed in detail in U.S. patent application Ser. No. 07/976,941,entitled, "Method for the Production and Transmission of EnhancedMultimedia Information," having a filing date of Nov. 16, 1992, assignedto the assignee in the present invention, and that patent application isincorporated by reference in this application. The following paragraphsalong with the accompanying figures will provide the details regardingthe optimization method and how it will be used advantageously toprovide an enhanced interactive multimedia system.

The purpose of the IMM 42 is to provide maximum interactivity while atthe same time providing maximum retention of the program model. It isalso important that there be minimum transit time for the interactivitywhile the information has maximum replication. Therefore, it is veryimportant that the program model psychographic parameters be welldescribed. For example, the spatial, color, temporal, audio response,material concept, contention perception all should be very welldescribed and defined in the program model.

Referring now to FIG. 3 what is shown is a first embodiment of a flowchart for providing an enhanced interactive image that utilizes theprinciples of the present invention. The flow chart 100 comprises thesteps of providing a program model to a separator. The separator 102will divide the information into primary and secondary layers ofinteractive multimedia information. The separation is automatic and canbe accomplished in a variety of ways. For example, the layers can beseparated by production sources. In another example, separation can beaccomplished through key coding the layers. In yet a third example, thelayers can be spatially separated or separated by the various colors.Finally, layers of information could be separated by a filteringprocess.

The primary layers are provided to the compression generation block 104.There are a variety of ways that the multimedia data can be changed orgenerated to use less bandwidth. For example, compression algorithms ortheir equivalents could be utilized to reduce the bandwidth used. Inaddition generators, such as in a tone generator system, could beutilized to reduce the bandwidth required. Finally, key coding systemscould be utilized to reduce bandwidth use. Such systems will bediscussed in more detail later in the specification.

In this embodiment, the primary layer is provided to an encoder wherethe primary layer is prepared for transmission (block 108). The primarylayer is then decompressed (block 110). Thereafter the primary layer ofinformation is decompressed (block 110). The primary layer is thendecoded and mixed with the secondary layer of data information (block112) to provide an enhanced interactive multimedia image (motion, videoor other) or data to a display.

Similarly, the secondary layer is compressed through block 114, encoded(block 116) and then transmitted to block 118 to decompress and mixblock 110. The two signals (primary and secondary) are then sent todisplay 112.

In this embodiment, for example, in the optimization of video images,the primary layer can be the foreground image, the secondary layer canbe background information. Through the use of this type of optimizationtechnique multimedia information can be enhanced while at the same timeutilizing significantly less bandwidth.

To more fully understand this feature refer now to the. followingdiscussion. In a typical interactive multimedia system the informationis all sent along one layer. The information that can then betransmitted is limited by the bandwidth of that layer.

In the prior art, the interactive multimedia information that could betransmitted along typical networks or transmission paths that are verylimited because, for example, in the case of vide images the bandwidthis not adequate to provide a high quality image to a display.

Hence, in the present invention, by separating the multimediainformation into primary and secondary layers and thereafter compressingthe more important information utilizing well known compressionalgorithms, a system is described that can produce enhanced interactivemultimedia information that easily be transmitted over existingnetworks.

To more fully describe the psychographic enhancement feature of thepresent invention refer now to FIG. 4 which shown the variouspossibilities from a particular program model. The program model isprovided to the separator 42 of the multimedia system.

Psychographic enhancements are critical to the improvement ininteractive multimedia transmission and reception enhancements in thecontext of the present application is information that is nottransmitted but operates on, operates in conjunction with, or operatesas a supplement to transmitted multimedia information. There are threeseparate categories that will be described that relate to psychographicenhancements.

The first category will be described as a cross correlation between theinformation that is being transmitted and being enhanced due to thepresence of information that is not transmitted. Dithering of image isan example of this in that dithering masks artifacts of images that arepresent and that improves the image quality. This type of system doesnot remove the artifacts but actually just masks imperfections. A secondexample in the audio arena where secondary audio materials such as asound of an ocean or the like which might mask problems in the audioquality of the primary sound (voice, music or the like).

The second category is where the signal is actually changed without theuse of any control signal; for example, through interpolation or someother technique. The typical example of that is a graphic equalizer inwhich certain frequencies are enhanced depending on the range of theparticular device. Another example of the second category is tofrequency or amplitude compress a certain signal so as to furtherenhance the frequencies that are being transmitted. It is also known touse various filters to sharpen or provide certain information that willactually modify the signal without controlling it per se.

Finally, the third category is using the primary and secondaryinformation to drive the other generators that might be present withinthe multimedia system. This can be utilized to either enhance themultimedia information or enhance the program model. An example of thisis the use of real-time graphics frequency spectrum displays to enhancea music juke box type of program model.

As is seen in FIG. 4, the primary multimedia information layer can becompressed to reduce the bandwidth utilizing well known algorithms. Itis also seen that the signal can be replaced by a generator that isresponsive to the primary/secondary layers signals. Finally, a key codecould be used to cause information to be provided from a look-up tableor the like.

Although all of the above methods provide advantages in accordance withthe present invention, key coding has some additional non-obviousadvantages when utilized in the optimization system of the presentinvention. In the following paragraphs the use of various key codingsystems will be described generally along with their attendantadvantages.

Typically, when looking at an interactive multimedia information signalthere are several components of that information. The first component isthe data or the multimedia information itself that is being conveyed.The second component is referred to as program model dynamics. That isthe changes that occur in the interactive multimedia information due tofor example, a fade that allows for a transition from one scene in thegraphics or video image to another. Conversely, if you want to wipe awayan image there is information associated with the multimedia data thatwould call out for that transition to change efficiently.

Finally, the third category of interactive multimedia information iswhat will be referred to in this specification which will allow aparticular device or system to go from one category to another. In atypical interactive multimedia information system all this informationis required to adequately transmit such information.

In its simplest form, a key has an identifiable code which dictates thecommands on the other side of the device. The clearest example of such akeying system would be the very simple dual tone multi-frequency (DTMF)signal. This type of signal can be used in the telecommunications areato provide keying for low bandwidth protocol. These keys would thencommand a code table on the side of the network to provide certaininformation about the multimedia information to be displayed withoutrequiring actual transmission of the multimedia information.

A more specific version of this type of key coding is what will bereferred to in this specification as control information keying. What ismeant by controlled information keying is where a key code is utilizedto access particular types of commands which can then be used to controlother items on the other side of the network.

Such a table would then be utilized to access a certain set ofmultimedia information in the network. A final version of key codingwill be called program branching keying is described by each of the keysrepresenting a certain branch identification. Thus in this type of keycoding the key is cross referenced to a particular branch of theinteractive multimedia program where each of the branches allowsplurality of functions or commands to be accessed in order to replicatethe program model.

The important feature that is provided by all of these types of keyingcoding arrangements is that information already present on the networkcan be utilized. Therefore, the processing power inherent in the networkor the system being accessed can be utilized rather than having to haveto provide that processing power within the optimization system itself.

It is also important to develop means to improve the transmissionquality of the multimedia information, for example, the information maybe transmitted utilizing a typical transmission algorithm such a CCITTv.32, or Bell 202 with standard communication file data transferprotocols. The interactive multimedia information could also utilizespecialized protocols that are optimized for the particular interactivemultimedia information that is to be transmitted. In so doing thealgorithm for the compression algorithm can be interactively matrixedwith the transmission algorithm to provide the highest qualityinformation with the maximum interactivity with the minimum transmissionline.

Referring now to FIG. 4, what is shown is a flow chart that shows thecooperation of the transmission algorithm with the compression algorithmto produce a high quality multimedia image. The flow chart comprisesproviding a program model in which the primary and secondary layers areseparated (block 202). The primary layer is compressed and encoded(blocks 204 and 206).

A control element (block 210) is utilized to control a

compression matrix and a transmission matrix. These two matricescomprise a plurality of compression algorithm and transmission algorithmrespectively that are interactively controlled such that as the variousalgorithms are detected the quality of the multimedia information andthe speed of the transmission are interactively determined.

The quality of the information could be determined manually or throughthe use of some control circuitry. It should be understood that thesesame matrices could also be used on the secondary layer. It was notshown or described for purposes of clarity or simplicity.

Referring now to FIG. 6 what is shown is a block representation of amatrix of compression algorithm with transmission algorithm that couldbe utilized in accordance with the present invention. The circles 302aligned in the vertical direction are the compression algorithms. Therectangles 300 aligned in the horizontal direction are the transmissionalgorithms.

For example the compression algorithms could be JPEG, a generator withMusical Instrumental Digital Interface (MIDI), and a key for a weathermap background. Similarly, the transmission algorithms could beoptimized for Joint Photographic Expert Group (JPEG), data compressionfor MIDI, or DTMF for key transmission type algorithms. To provide thehighest quality multimedia information while at the same time utilizingminimum bandwidth the different algorithms can be selected in aninteractive manner.

Hence, a first compression algorithm could be selected along with thefirst transmission algorithm. The multimedia information is reviewedeither for image or audio quality than a second compression algorithm isselected. The multimedia information is reviewed and if the quality isnot acceptable then a second transmission algorithm is selected. Thequality of the information is reviewed. This process is repeated untilthe highest or desired quality multimedia information and interactivityspeed are provided.

The multimedia information derived from the compression/transmissionalgorithms can be analog or digital in nature. However, in a digitalsignal there are certain other features that can be taken to advantagethat can be utilized in accordance with the present invention.

It is known that digital data information is typically sent in a filewhich specifies certain parameters of that data and the data informationitself and within the data information itself is information which maynot change for a certain set of files. In the case of an image file, theheader information may specify the dimensions, pixel depth, and certainother features of the particular image. This file may take up a much astwenty percent of the data file.

Conversely, in a file such as MIDI music file which comprise a pluralityor a series of music notes, the header may include instrumentinformation, related information and characteristics of that particularfile. In both of the above mentioned examples, the header informationmay not change, through the use of the optimization method the amount ofinformation may be significantly reduced over time.

Hence, in the case of the image file, the header could be sent firstwith no compression or with lossless data compression as the secondaryfile because it will always remain the same. The date file itself canthen be compressed down to its smaller size.

Another method for enhancing the psychographic parameters is to providesome form of error detection and adjustment. As has been mentionedbefore the detection and adjustment can be accomplished viainterpolation of the error. An alternative method of error corrector isthrough an error correction/transmission algorithm. What is meant bythis, is relating the transmission to the compression to enhanceinteractivity.

In this type of system before the file is sent the base file iscompressed and then decompressed. This decompressed file is called anexpanded compressed base file. The expanded compressed base file is thencompared to the original base file and an error file is then developed(the error file being the difference between the base file and theexpanded compressed base file). The error file is compressed and sentalong with the compressed version of the base file down the line. Thesefiles are then combined and decompressed to provide an enhanced image.

Referring now to FIG. 7 it is seen that a data file utilizing thistechnique could initially be separated into primary and secondarylayers. The primary layer could be compressed using a first compressionalgorithm, the header could be sent first along a first transmissionpath and the compression signal could be sent along a secondtransmission path.

Therefore, the amount of storage necessary for the file is significantlyreduced through secondary compression techniques. This information canthen be transmitted or stored across the network rather than having tohave all that information stored within a particular device within theoptimization system.

It should be understood by one of ordinary skill in the art willrecognize that the number of algorithms is not limited to the numbershown in the figures. In addition it should be recognized that the orderor the selection of the algorithms could be changed and that would bewithin the spirit and scope of the present invention.

The present invention has been discussed in terms of compressing theprimary layer or layer and by compressing and transmitting that primarylayer in a particular way the interactivity of the system is enhanced.It should be understood that it may be equally important to enhancesecondary layers to produce the same effect.

Therefore, it may be important to enhance the secondary layer, it may beimportant to enhance the primary layer or it may be important to enhanceboth. Therefore, the present invention through the use of compressionand transmission algorithms and through the psychographic enhancement ofthe program model can enhance interactivity of a multimedia system.

It should also be understood that the function of the compression andtransmission algorithms can also be done through other means; forexample, a signal generator could be used to provide the sameinformation. That is, a signal generator responsive to a particularlayer or layer of information could be utilized to provide thatinformation or some level of information that is representative of thatlayer. For example, a tone generator responsive to a signal from thesecondary layer to provide the tone that would be representative of thatsecondary layer.

Conversely, some type of graphics generator could be utilized to respondto that same type of signal to provide a certain type of graphic imagein a video system. Finally, it should also be understood that thepsychographic parameters can be adjusted by human operator or in thealternative can be adjusted or modified by an automatic means.

The optimized multimedia information from the program source material isthen processed by the MCPS 44 (FIG. 2) to be utilized interactively withIMDs 52. Referring now to FIG. 8 what is shown is the preferredembodiment of a MCPS 44. The MCPS comprises a distributed computingarchitecture 500. The distributed computing architecture 400 includes amaster node 502 that has several server nodes 504 which in turn areconnected to several ports 506. When connected via the existing networkssuch as to a switched telephone network and then on to connection to theIMD 52 which has its own processing storage and computing structure theentire network can be operated as a massive distributed computingenvironment. This environment shares all dimensions of computing,storage, transmission and peripheral resources (printing, productordering, mailing functions, etc. ). This type of computing architecturewould include dynamic port allocation and would include incrementalfailure characteristics to allow for robustness of the MCPS 44.

It also is very important in this system architecture to have aninteractive multimedia device 52 which will allow for the receipt ofhigh quality multimedia information from the system architecture. TheIMD 52 can be a printer that has been attached to a video display. Itcan also be a facsimile machine which could be utilized to receivedenhanced facsimile-like information. The IMD 52 can also be utilizedwith a television monitor to provide enhanced audio, video and graphicinformation thereto. Other connections to common household devices suchas a VCR and household heating control systems can also be utilized bythe IMD 52.

FIG. 9 is a preferred embodiment of an interactive multimedia decoder(IMD) 52A. The IMD 52A comprises several components. The telephone lineis coupled to a multimedia modem 522. The multimedia modem is coupled toa multimedia memory 524 which can be an expandable dynamic random accessmemory (DRAM) 524. The multimedia modem chip 522 provides data to amultimedia decompressor device 526. The multimedia memory provides datato graphics/character generator 536, speech generator 538 and musicsynthesizer 540.

In addition, the output of the generators 536, 538 and 540 are providedto a video control chip 532. Video control 532 provides signals to astandard television display and receives signals from a standardtelevision source. The multimedia modem 532, the multimedia memory 524,the multimedia decompressor 526, the multimedia digital/audio control528, the video control 532 and music synthesizer 544 are all ultimatelycontrolled by an interactive control interface 530 which manages theoperation of all of the above elements. The video control 528 is coupledto a standard telephone keypad input or for a television remote-typedevice or a special IMD remote can be utilized in a variety of wayswhich will be discussed in detail hereinafter.

Personalized and demographic information (such as the age, race, sex andother personal characteristics ) of the user along with the technicalinformation associated with the IMD 52 (serial no., number ofgenerators, type etc. ). Upon connecting the IMD 52 to the MCPS 44, theIMD 52 can both forward this information either at the beginning of thesession or any time afterward. The information can then be updatedthrough the MCPS 44 or directly IMD 52 through selection using a keypador remote control or by receiving data from the MCPS 44.

Another mode of operation is the delivery of multimedia during theperiod when a communication network is not in use. In this way, largeamounts of multimedia information can be efficiently transmitted andstored in the IMD 52 for IMD for later review and enhancedinteractivity.

The function of each of the different components in a preferredembodiment is described in a summary fashion below.

MULTIMEDIA MODEM 522

a. Responsible for all communications between standard phone line,optional serial port, interface to multimedia memory, multimedia decode,audio control, and processor control modules.

B. Supports standards protocol for half-duplex, full duplex, andhalf-duplex high speed operation.

C. On-chip encode/decode capability, D/A, A/D for voice, facsimile, anddata functions.

D. Dual tone multi-frequency (DTMF) detect and generation.

E. Auto-detect voice/facsimile/data switch for transparent modetransition.

F. Incorporates controller unit with binary file transfer, facsimile,data, and voice modes, and optional proprietary multimedia processorcontrol optimized protocol firmware.

G. Firmware allows IMD to use multimedia modem to perform callprocessing function including telephone call dialing and connection,unattended receipt of data and fax among other functions.

MULTIMEDIA MEMORY 524

a. Nominal DRAM or VRAM for image mixing/processing, and auxiliarymultimedia data store.

B. Nominal ROM for resident IMD control program.

C. Optional co-resident DRAM for multimedia data store and program/datastore.

D. Optional non-volatile storage (extendible ).

E. Memory control unit for VRAM/ROM/DRAM and non-volatile storage.

MULTIMEDIA DECODE 526

a. Responsible for real-time decompression of images transferred to orstored in the IMD 52 with multiple algorithm coding.

B. On chip inverse discrete cosine transform processor.

C. Reverse quantizer decoder/tables.

D. Built-in zoom, pan, chroma key, mix from compressed data incorporatesinterfaces to video data bus, multimedia memory, multimedia modem, videocontrol, and microprocessor control sections.

VIDEO CONTROL 532

a. Responsible for all IMD 52 video mixing, enhancements, and displayfunctions.

B. Pixel processor for mix, zoom, pan, chroma key, transform on pixeldata, transitions.

C. Graphics processor for figures (e.g., rectangles with color fill)generation, sprites, text with foreign characters, and scrolling.

D. Digital to analog conversion, analog to NTSC, NTSC video plus stereoaudio to RF.

GRAPHICS/CHARACTER, SPEECH GENERATOR, MUSIC SYNTHESIZER 536, 538 AND 540

a. Responsible for enhancing received analog/digital audio, musicsynthesis generation, and overall analog mixing and audio effects.

B. Incorporates decoding burden.

C. Sampled instrument synthesis from compressed MIDI input.

D. Built-in micro-controller for multi-task generation.

E. Dual analog source mix, digital audio and synthesizer mix, analogaudio control (volume, bass, treble, balance) for output to analogleft/right audio.

INTERACTIVE MULTI-TASK PROCESSOR 530

A. Responsible for multi-task execution of resident and downloaded IMDcode for operation in conjunction or independently of the MCPS.

B. Master/slave microcontroller architecture for multi-task control ofcommunications, multimedia memory, multimedia decode, digital videocontrol, digital audio/synthesis, and interface management.

In a preferred embodiment, the IMD 52 (FIG. 2) will be utilized with atelevision monitor to transmit and receive multimedia information.

FIGS. 10, 11 and, 12 show different embodiments of an IMD 52. FIG. 10shows an IMD 52B which has an output to a printer 501 and a telephonehandset 540. The printer 501 then is utilized to print multimediainformation responsive to the telephone handset.

FIG. 11 shows an IMD 52C that has an output to a facsimile 700 and willprint facsimile-like information received from the system architecture40. This so-called FAX IMD 52C includes a facsimile machine internalthereto.

FIG. 12 shows an IMD 52D that has an output to a television 53 and alsohas an output connected to a fax machine. This will allow for a viewerto view the incoming facsimile messages and other multimedia informationand then print them via a fax machine.

Yet another preferred embodiment of the present invention comprises aMCPS 44 which incorporates a standard voice response computer orinteractive voice response system or the like to perform digital oranalog voice functions of the MCPS 44. In addition this apparatus wouldalso perform the DTMF functions for MCPS program control. To perform themultimedia information transmission to the IMD 52 and to control theswitching between voice data , fax the system would utilize amodification to standard voice response systems, interactive voiceresponse system or a modification thereof. By utilizing the appropriateIMD 52 protocol reconnection to a standard telephone through the abovementioned modified systems. Accordingly, either one standard telephoneline or a plurality of telephone lines operating in parallel may beutilized for transmission from the MCPS 44 to the IMD 52.

Another critical feature of the IMD is to have a remote control thatwill work in conjunction with the TV or other display or the like toprovide enhanced multimedia information. To more fully explain thisfeature refer now to FIG. 13. The remote control 900 looks much like atelephone keypad. It has the numerals (0-9) and symbols (*-#) that arepart of an ordinary telephone keypad. It includes an enter key 902 thatis typically utilized to change information or change channels in thecase of a television set. The control 900 would also include a volumekey 904 and a channel or memory select key 906, a connect key 908 and amultimedia toggle button 910. It could also include a credit card slot912. This credit card slot 912 would be utilized by the viewer to allowfor the purchase of certain items directly while viewing the televisionscreen. Finally, the control 900 includes special effect keys 916, forexample, for allowing for the browsing of a multimedia directory whilesimultaneously displaying a picture in the picture of the currentbroadcast TV channel.

The remote control 900 could utilize a radio frequency signal or audiosignal to interact with the receiver and/or IMD 52 for the control ofthe IMD, control the selection of multimedia information, and for thecontrol of other household devices. It is also known that a moreconventional remote control could be utilized such as one that controlsa VCR or a television and its use would be within the spirit and scopeof the present invention.

Another feature of the above-identified system architecture 40 of FIG. 2is that the program information can be linked to the network to providefor enhanced interactivity and program quality. For example, the programsource can follow or be synchronized with the cable or broadcast feed toprovide for enhanced distribution of program information. One specificexample of this to take a new program, like CNN broadcasting, whichconsists of several short news items. For each news there could bestored large archives or program material related to a particular story.Hence, through multimedia interaction with for, example, the remotecontrol 900 the stored program material can be accessed. This wouldallow the user to review the material in whatever detail was desired.

Another example, is in an advertising feed if a particular item wasadvertised, there would be the ability to refer to more detailedinformation about the product.

Yet another example of such a system is to have one service linked toanother. An advertisement could be linked to a means for ordering theparticular product being advertised including the placement of a phonecall by the IMD to the desired telephone number. In so doing, the linkedsystem allows for additional services to be accessed.

The important requirements for this linked feature is that there is aknowledge of the contents of the system program requirements and thereis a knowledge of where the user is in the program. For example, an IMDcould be used for channel program identification in which the telephonecall is linked to a particular channel that has been selected. Anotherexample is link demographics for targeted interactive advertising.hence, in an advertisement for a diaper, for example, there would be thefacility to access advertisements for related items such as baby powder,baby oil or the like through the use of this linked approach.

It should be understood that the IMD 52 itself could be utilized as acall processing system. Finally, it should be understood that therecould be multiple MCPS 42 to provide for very large scale callprocessing.

Although the present invention has been described in accordance with theembodiments shown in the figures, one of ordinary skill in the artrecognizes there could be variations to the embodiments and thosevariations would be within the spirit and scope of the presentinvention. Accordingly, many modifications may be made by one ofordinary skills in the art without departing from the spirit and scopeof present invention, the scope of which is defined solely by theappended claims.

What is claimed is:
 1. An interactive multimedia device for use in acommunication network comprising:a multimedia modem means for receivinginteractive multimedia information from the communication network;memory means for receiving interactive multimedia information from themultimedia modem means; means for compressing at least a portion of theinteractive multimedia information received from the multimedia modemmeans and the memory means; means for decompressing at least a portionof the interactive multimedia information; means for transmitting theuncompressed portion of the interactive multimedia information receivedfrom the communication network; means for mixing the decompressedportion with the uncompressed portion and providing an output; means forproviding the output of the interactive multimedia information to adisplay; and means for interactively controlling the multimedia modemmeans, compressing means, decompressing means, mixing means andproviding means to provide interactive multimedia information to thedisplay.
 2. The device of claim 1 in which the interactive controllingmeans comprises a microprocessor.
 3. The device of claim 1 in which themixing means comprises means for receiving the compressed portion andthe uncompressed portion.
 4. The device of claim 1 which furthercomprises means for dual tone multi-frequency detection (DTMF) from thecommunication network.
 5. The device of claim 1 in which thecommunication network comprises a telephone network.
 6. An interactivemultimedia device for use in a communication network comprising:meansfor receiving interactive multimedia from the interactive multimediainformation; means for compressing a first portion of the interactivemultimedia information; means for decompressing the primary layer of theinteractive multimedia information; first transmitting means fortransmitting the first portion along a first transmission path; secondtransmitting means for transmitting a second portion of the interactivemultimedia information along a second transmission path, the secondportion of the interactive multimedia information being uncompressed;means coupled to the first transmitting means for decoding the firstportion of the interactive multimedia information; means coupled to thedecoding means and the second transmitting means for mixing the decodedfirst portion and the uncompressed second portion; and means coupled tothe mixing means for printing the interactive multimedia informationreceived from the mixing means; and means for interactively controllingthe receiving means, the memory means, the compressing means, the firsttransmitting means, the second transmitting means, the decoding means,the mixing means and the printing mans to print multimedia information.7. The device of claim 6 in which the communication network comprises atelephone network.
 8. An interactive multimedia device for use in acommunication network comprising:multimedia modem means for selectinginteractive multimedia information; the multimedia modem means furthercomprising: means for separating the multimedia into primary andsecondary layers based upon a program model; memory means for receivinginteractive multimedia information from the multimedia modem means;means for enhancing the primary layer from the multimedia modem means inaccordance with the program model to enhance interactivity; means forcompressing the primary layer received from the enhancing means; meansfor decompressing the primary layer of the interactive multimediainformation; means for transmitting an uncompressed portion of theinteractive multimedia information received from the communicationnetwork; means for mixing the decompressed primary layer with theuncompressed portion and providing an output; means for providing theoutput of the interactive multimedia information to a display; and meansfor interactively controlling the multimedia modem means, compressingmeans, decompressing means, mixing means and providing means to provideinteractive multimedia information to the display.
 9. The device inaccordance with claim 8 in which the multimedia modem means furthercomprises:means for providing the secondary layers through psychographicparameters; and means for combining the primary and secondary layers.10. The device of claim 8 in which the enhancing means furthercomprises:means for encoding the primary layer in accordance with afirst compression algorithm.
 11. The device of claim 8 in which theenhancing means comprises:means for encoding the primary layer inaccordance with a first transmission algorithm.
 12. The device of claim8 in which the separating means comprises keying the primary andsecondary layers to the psychographic parameters.
 13. The device ofclaim 9 in which the separating means comprises separate productionsources for providing the primary and secondary layers.
 14. The deviceof claim 8 in which the separating means comprises means for spatiallyseparating the primary and secondary layers in accordance with thepsychographic parameters.
 15. The device of claim 8 in which thecommunication network comprises a telephone network.
 16. An interactivemultimedia device for use in a communication networkcomprising:multimedia modem means for selecting interactive multimediainformation; the multimedia modem means further comprising: means forkeying multimedia information into primary and secondary layers basedupon a program model; memory means for receiving interactive multimediainformation from the multimedia modem means; means for enhancing theprimary layer from the multimedia modem means in accordance with theprogram model to enhance inactivity; means for compressing the primarylayer received from the enhancing means; means for decompressing theprimary layer of the interactive multimedia information; means fortransmitting an uncompressed portion of the interactive multimediainformation received from the communication network; means for mixingthe decompressed primary layer with the uncompressed portion andproviding an output; means for providing the output of the interactivemultimedia information to a display; and means for interactivelycontrolling the multimedia modem means, compressing means, decompressingmeans, mixing means, and providing means to provide interactivemultimedia information to the display.
 17. The device of claim 16 inwhich the communication network comprises a telephone network.